Speech emotion recognition based on listener-dependent emotion perception models

This paper presents a novel speech emotion recognition scheme that leverages the individuality of perception. Most conventional methods simply poll multiple listeners and directly model majority decision as perceived emotion. However, perception varies with listener, which forces their single models to create complex mixtures criteria. In order mitigate this problem, we propose majority-voted framework constructs listener-dependent (LD) models. The LD can estimate not only listener-wise emotion, but also by averaging outputs Three models, fine-tuning, auxiliary input, sub-layer weighting, are introduced, all inspired successful domain-adaptation frameworks in various processing tasks. Experiments on two emotional datasets demonstrate proposed approach outperforms recognition.